# 3D Animation

GaussianSpeech is an innovative method capable of synthesizing high-fidelity animated sequences from speech signals to create realistic, personalized 3D head avatars. The technology combines speech signals with 3D Gaussian drawing techniques to capture human head expressions and detailed movements, including skin wrinkling and finer facial motions. Key advantages of GaussianSpeech include real-time rendering speed, natural visual dynamics, and the ability to exhibit a variety of facial expressions and styles. The underlying technology involves the creation of large-scale, multi-view audio-visual sequence datasets and the development of audio conditional transformation models that can directly extract lip and expression features from audio input.

Vmotionize is a leading AI animation and 3D animation software capable of transforming videos, music, text, and images into stunning 3D animations. The platform offers advanced AI animation and motion capture tools, making high-quality 3D content and dynamic graphics more accessible. Vmotionize revolutionizes the way independent creators and global brands collaborate, enabling them to bring their ideas to life, share stories, and build virtual worlds through AI and human imagination.

DrawingSpinUp is an innovative system that converts single character drawings into 3D animations. This technology addresses the challenges of amateur character drawings in appearance and geometry by removing view-dependent outlines and re-rendering along with a skeleton-based refinement deformation algorithm. It not only enhances the visual quality of character drawings but also breathes dynamic life into them, allowing for free rotation, jumping, and even street dance performances.

Animate3D is an innovative framework for generating animations for any static 3D model. Its core ideas include two main parts: 1) Propose a new multi-view video diffusion model (MV-VDM) based on multi-view rendering of static 3D objects and trained on our large-scale multi-view video dataset (MV-Video). 2) Introduce a framework combining reconstruction and 4D score distillation sampling (4D-SDS) that utilizes multi-view video diffusion prior to animate 3D objects. Animate3D enhances spatial and temporal consistency through the design of new spatiotemporal attention modules, and maintains the identity of static 3D models through multi-view rendering. Additionally, Animate3D proposes an effective two-stage process for generating animations for 3D models: First, it directly reconstructs motion from the generated multi-view video, and then refines appearance and motion through the introduced 4D-SDS.

SC-GS is a novel representation technique that utilizes sparse control points and dense Gaussian functions to represent the motion and appearance of dynamic scenes, respectively. It learns a compact 6-degree-of-freedom (6DOF) transformation basis using a small number of control points, which can be locally interpolated through weight interpolation to obtain a motion field for the 3D Gaussian function. By employing a deformation MLP to predict the time-varying 6DOF transformation for each control point, the method reduces the complexity of learning and enhances its capacity, achieving coherent spatiotemporal motion patterns. Simultaneously, it conjointly learns the 3D Gaussian function, the canonical space positions of the control points, and the deformation MLP to reconstruct the 3D scene's appearance, geometry, and dynamics. During training, the position and number of control points are adaptively adjusted to suit the motion complexity of different regions. A constraint loss function that enforces spatial continuity and local rigidity is also adopted. Thanks to the explicit sparsity of the motion representation and the separation of appearance, this method enables user-controlled motion editing while preserving high-fidelity appearance. Extensive experiments demonstrate that the proposed method outperforms existing approaches in new view synthesis and high-speed rendering, and supports novel applications of user-controlled motion editing with preserved appearance.

_MOVE API is capable of converting videos capturing human motion into 3D animation assets. It supports converting video files to the usdz, usdc, and fbx file formats while offering a preview video. Ideal for integrating into production workflow software, enhancing motion capture capabilities, or creating novel experiences.

This product is based on a diffusion model framework, designed to generate 3D motion sequences of humans from a single image. Its core components include learning priors about the unseen parts of the human body and clothing, and rendering new body poses with appropriate clothing and textures. We train the model in the texture map space to make it invariant to pose and viewpoint, thus more efficient. Additionally, we develop a diffusion-based rendering pipeline controlled by 3D human poses that produces realistic human renderings. Our method can generate image sequences that align with 3D pose targets while visually resembling the input image. The 3D control also allows for generating various synthetic camera trajectories to render human figures. Experiments demonstrate that our approach can generate image sequences of continuous motion and complex poses, outperforming previous methods.